class: center, middle, inverse, title-slide .title[ # Transformation of Covariates: Splines ] .subtitle[ ## Lecture 7 ] .author[ ### Manuel Villarreal ] .date[ ### 09/30/24 ] --- ### Linear Splines - A common type of functions used in linear models are known as step functions. -- - As it name suggests, step functions are rules where the value assigned changes abruptly. -- - For example, we could have a linear function that assigns the value 0 to participants that are younger than 21 years of age and the value of 1 to anyone that is older than 21. -- - Using step functions will allow us to stratify groups based on continuous variables. -- - Furthermore, we could modify these functions in order to have something that resembles an interaction. -- - Let's look at an example with a memory experiment. --- ### Example: Memory - Researchers carried out an experiment to study how memory changes as a function of age in Young and Elderly populations. -- - The study consisted of multiple memory tasks that the researchers have used to calculate an index that reflects the performance of each participant in the task. -- - Let's first look at the data file and make a plot. -- Open the file `in-class-08.Rmd` and follow the instructions. --- ### Example: memory - Answer questions one and two from the in class activity -- <div class="datatables html-widget html-fill-item" id="htmlwidget-220f7df309fbabd3d7ae" style="width:100%;height:auto;"></div> <script type="application/json" data-for="htmlwidget-220f7df309fbabd3d7ae">{"x":{"filter":"none","vertical":false,"data":[[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142,143,144,145,146,147,148],[2.7,2.19,2.91,1.45,1.87,0.85,2.72,2.32,3.95,3.31,3.1,2.68,3.17,2.99,1.75,1.76,1.41,4.88,2.17,4.13,1.17,1.17,2.46,1,1.53,2.1,0.45,3.96,0.23,3.82,0.21,0.26,1.26,3.86,0.51,1.58,1.63,1.38,0.31,3.87,2.87,3.99,4.99,0.3,3.34,1.96,2.41,3.93,1.94,3.54,3.17,2.1,-0.71,3.89,3.49,3.35,1.98,3.79,3.06,0.8,4.14,3.15,3.69,0.83,0.53,3.56,3.38,3.66,2.32,0.51,2.23,-0.27,0.43,3.7,4.17,2.93,0.8100000000000001,1.69,0.16,2.94,3.97,2.86,3.72,2.06,2.45,3.73,4.17,2.98,1.01,3.03,1.2,2.16,-0.25,3.84,3.19,2.13,0.42,2.48,2.9,2.99,-0.46,3.54,1.61,0.92,1.64,4.01,1.99,1.77,0.98,2.8,1.99,4.26,1.21,1.36,3.06,1.47,-0.21,1.59,3.1,3.3,0.35,3.16,2.2,0.65,4.09,0.63,4.1,1.53,0.72,3.82,1,0.61,2.59,4.45,3.31,2.48,0.27,1.72,4.12,2.25,0.67,1.34,3.05,3.1,0.88,2.57,3.38,1.97],[43,59,34,55,56,71,37,30,31,31,22,51,39,28,60,66,55,37,53,39,62,66,44,67,55,43,71,27,65,21,74,70,63,21,65,59,61,59,69,23,51,22,34,72,23,62,57,37,52,34,32,53,75,40,39,40,63,30,38,58,35,41,26,68,70,23,40,41,51,73,53,72,75,34,24,36,62,54,74,47,20,40,35,52,47,29,32,27,63,48,62,58,73,34,49,48,70,55,51,46,75,33,55,72,64,39,54,57,61,24,60,44,60,61,38,63,73,62,52,50,74,42,46,73,24,71,39,64,68,42,66,66,57,25,38,54,72,49,42,42,65,60,29,30,71,59,39,55]],"container":"<table class=\"cell-border stripe\">\n <thead>\n <tr>\n <th>id<\/th>\n <th>performance<\/th>\n <th>age<\/th>\n <\/tr>\n <\/thead>\n<\/table>","options":{"pageLength":5,"columnDefs":[{"className":"dt-right","targets":[0,1,2]},{"name":"id","targets":0},{"name":"performance","targets":1},{"name":"age","targets":2}],"order":[],"autoWidth":false,"orderClasses":false,"lengthMenu":[5,10,25,50,100]}},"evals":[],"jsHooks":[]}</script> --- ### Example: memory - Now that we have looked at the data let's start by plotting performance as a function of age and see if we can find any trends (question 3 `in-class-08.Rmd`). -- <img src="data:image/png;base64,#lecture-07_files/figure-html/unnamed-chunk-2-1.png" width="42%" style="display: block; margin: auto;" /> --- ### Example: memory - From the plot it seems like the memory performance of participants in the experiment decays with age. -- - First let's look at a linear model that includes age as a predictor of memory performance and look at the residuals to see if a linear regression is enough. -- .pull-left[ <img src="data:image/png;base64,#lecture-07_files/figure-html/unnamed-chunk-3-1.png" width="72%" style="display: block; margin: auto;" /> ] .pull-right[ <img src="data:image/png;base64,#lecture-07_files/figure-html/unnamed-chunk-4-1.png" width="72%" style="display: block; margin: auto;" /> ] --- ### Example: memory - From the data it seems like participants between 20 and 40 years of age get similar memory performance scores. -- - However, the scores for people who are over 40 seem to decrease rapidly in our sample. -- - What can we do with this? -- - The effect of age seems to change abruptly after 40 years old. -- - Age does not seem to affect the memory performance of participants younger than 40 but it decays for people that are older. -- - One possible solution would be to stratify using an age group, for example, we could set our two groups to people who are older or younger than 40 years old. --- ### Example: memory - Another way to look at the problem is to think of the effect of age on memory performance as a step function. -- - For example, we know that memory performance is constant as a function of age for people who are younger than 40 years old. -- - That means that we can describe the performance of people younger than 20 with just one parameter that is not associated with age. `$$\text{Performance}_i = \beta_0 + \epsilon_i, \text{if Age}_i \leq 40$$` -- - And then for participants that are older than 40 we would like the model to have the following form `$$\text{Performance}_i = \beta_0 + \beta_1\text{Age}_i + \epsilon_i, \text{if Age}_i > 40$$` --- ### Example: memory - Now because of how our data looks, we want to make sure that the only difference between a person who is 40 years old, and a person that is 41 years old is equal to `\(\beta_1\)`. -- - All this means that we want the following equation to be true: `\begin{equation} \beta_1f(\text{Age}_i) = \left\{ \begin{array}{lr} 0 & \text{if } \text{Age}_i \leq 40\\ \beta_1(\text{Age}_i - 40) & \text{if } \text{Age}_i > 40 \end{array} \right. \end{equation}` -- - Which means that `\begin{equation} \text{Memory_i} = \left\{ \begin{array}{lr} \beta_0 +\epsilon_i & \text{if } \text{Age}_i \leq 40\\ \beta_0 +\beta_1(\text{Age}_i - 40) +\epsilon_i & \text{if } \text{Age}_i > 40 \end{array} \right. \end{equation}` --- ### Example: memory - Therefore, we have that the difference between a person who is 40 years old and a person who is 41 is: `$$\mathrm{E}(\text{Memory}_i\mid \text{Age}_i = 40) - \mathrm{E}(\text{Memory}_i\mid \text{Age}_i = 41) =\\ \beta_0 - (\beta_0 +\beta_1(41 - 40))$$` `$$\mathrm{E}(\text{Memory}_i\mid \text{Age}_i = 40) - \mathrm{E}(\text{Memory}_i\mid \text{Age}_i = 41) = -\beta_1$$` -- - We can generate a variable in R that assigns a value of `\(0\)` to participants who are younger than `\(41\)` years old and a value of `\(\text{Age}_i - 40\)` to participants that are `\(41\)` years old or older. --- ### Example: memory - Add a new variable to the data frame named `age_spline` that assigns the value a value of `\(0\)` to participants who are younger than `\(41\)` years old and a value of `\(age - 40\)` to participants who are `\(41\)` years old or older (question 4 `in-class-08.Rmd`). -- - There are multiple ways to add a variable to a data frame. One that you can find in the cheat-sheets is: ``` r memory <- memory |> dplyr::mutate("age_spline" = ifelse(test = age > 40, yes = age - 40, no = 0)) ``` -- - This new column (variable) is our function of age denoted as `\(f(\text{Age}_i)\)`. --- ### Example: memory - Now we can fit a linear model that includes our transformed variable as a covariate. -- - In this case we do not need to add the variable itself, as the transformation is already taking into account the two types of effects we saw in the data (question 5 `in-class-08.Rmd`). -- ``` r lm_age_40 <- lm(formula = performance ~ age_spline, data = memory) ``` --- ### Parameter interpretation - Now we can interpret the estimated values of the parameters in our model (question 6 `in-class-08.Rmd`). -- - The estimated value of the parameter `\(\beta_0\)` was equal to 3.55. - The memory performance of participants who are 40 years old or younger was equal to 3.55. -- - The estimated value of the parameter `\(\beta_1\)` was equal to -0.1. - The memory performance of participants who are 41 years old or older decreases by 0.1 units on average per year. --- ### Predictions of a Spline model - Add the predictions of your new linear model to the plot of participant's memory performance as a function of age (question 7 `in-class-08.Rmd`). -- <img src="data:image/png;base64,#lecture-07_files/figure-html/unnamed-chunk-7-1.png" width="504" style="display: block; margin: auto;" /> --- ### Residuals - Plot the model residuals as a function of age (question 8 `in-class-08.Rmd`). -- <img src="data:image/png;base64,#lecture-07_files/figure-html/unnamed-chunk-8-1.png" width="504" style="display: block; margin: auto;" /> --- ### Autocorrelation - Plot the autocorrelation of the model's residuals (question 9 `in-class-08.Rmd`). -- <img src="data:image/png;base64,#lecture-07_files/figure-html/unnamed-chunk-9-1.png" width="504" style="display: block; margin: auto;" /> --- ### Evaluation: BIC - How can we evaluate our new model that uses a transformation of age? Well, like we have done before, we can compare models using the BIC (question 10 `in-class-08.Rmd`). -- - The estimated BIC for the model that includes only age was 275.1, while the BIC of the model that includes the transformation was 229.2. -- - The model that assumes that there is a change in the effect of age in memory performance only for those participants who are older than 40 accounts for the data better according to the BIC. --- ### Evaluation: Residuals - However, in this case we can evaluate the adequacy of the linear model assumptions (question 11 `in-class-08.Rmd`). -- - The residuals of the model that includes only age as a covariate show a bias as a function of age, with a general trend that increases for participants who are 20 to 40 years of age and the decreases as participants get older. -- - The model that includes the transformation doesn't show this patter, the residuals seem to be centered at 0 and have a constant variance across participant's ages. Additionally, the low autocorrelation of the residuals suggests that observations might be independent. --- ### What are linear splines? - The linear model that we just used is known as a linear spline. -- - A linear spline is a method where two or more straight lines are used to account for multiple linear patters in a data set. -- - Notice that what we did was basically join two independent lines together at a given point. -- - This point is known as a node, while the lines are known as splines. -- - In theory we could join any number of lines together in order to approximate any data patter. -- - Think about our example with the average temperature of two cities. -- - In theory we could glue together multiple lines that increase as we move towards the summer months with a line that decreases during the winter. --- ### Linear splines - Would a model that includes multiple straight lines be better than a model that uses a continuous transformation? -- - The answer (as is often the case in statistics) is: It depends. -- - For example, if we need more than 3 straight lines to account for the temperature data, but only two continuous transformations (like the cosine function in our example). -- - Then it might be the case that the continuous transformations would be better. -- - Regardless of the transformation that we use our approach would be the same. -- - First we fit the models and then we compare the residuals and use the BIC to select the model that better accounts for our data.